Method and device for aligning coordinate of controller or headset with coordinate of binocular system

ABSTRACT

The present disclosure provides a method and a device for aligning a coordinate of a controller or a headset with a coordinate of a binocular system. The method includes: obtaining pose data of the controller; acquiring real-time coordinate data in a binocular coordinate system during a repeating movement of the controller based on a light spot emitted from the controller; generating a motion track of the controller during the movement based on the real-time coordinate data and the pose data; obtaining a pose of the controller in controller coordinate system based on the motion track; obtaining a first angle between the positive direction of the controller coordinate system and a positive direction of the binocular coordinate system; and correcting the binocular coordinate system based on the first angle until the angle reaches zero. The method and device for aligning coordinates is easy to be implemented with low cost.

CROSS-REFERENCE TO RELATED APPLICATION

This Application is a continuation of International Application No.PCT/CN2017/096120, entitled “METHOD AND SYSTEM FOR ALIGNING COORDINATES,AND A VIRTUAL REALITY SYSTEM” filed Aug. 4, 2017, which claims benefitof Chinese patent application No. 201710278094.2 filed on Apr. 25, 2017,which is incorporated by reference herein in its entirety for allpurposes.

TECHNICAL FIELD

The disclosure relates to the field of virtual reality, mixed reality,and/or augmented reality, and more particularly to a method and devicefor aligning coordinate of controller or headset with coordinate ofbinocular system.

BACKGROUND

With the development of virtual reality (VR), augmented reality (AR) andmixed reality (MR), it is not only necessary to improve the content forwatching, but also to establish communicate between the virtualenvironment and the real world via various optical elements, sensors,and the like. Thus, the virtual environment can be operated andcontrolled by real devices in the real world.

VR is taken as an example herein. An existing VR interactive device mayinclude a head mounted display (or a headset), an internal or externaltracking transmitter or receiver (such as a stereo camera, an infraredreceiver or emitter, or a signal emitting receiver) and a hand-holdcontroller. The cooperation among the headset, the tracking transmitteror receiver and the controller can be used for capturing motion of theuser.

The method mainly applies Inside-Out (hereinafter referred to as IO) andOutside-In (hereinafter referred to as OI) to capture actions. Forexample, IO includes pose recognition applied by Leap Motion, Xhawk byXimmerse, and the like. OI is provided with Occulus Constellation, HTCVIVIE, Sony PSVR, and the like.

In the above scheme, the coordinate of an IMU (Inertial MeasurementUnit, which is used for measuring the three-axis pose angle or angularrate of an object, wherein the IMU module includes a gyroscope, anaccelerometer and a magnetometer) in the controller needs to be alignedwith the optical coordinate (the coordinate of device used for trackingand positioning the controller). At present, multi-light spot targetpose estimation is often adopted to directly estimate a pose of anobject through optical tracking in alignment scheme. Since multiplelight spots are adopted, it is necessary to arrange a plurality of lighttransmitters or receivers on headset. The light spots must be deployedin all directions according to the existing scheme, therefore thecomplexity is very high. The installation of the light transmitter orreceivers requires high precision and quite complex calibration, thecost is quite high and difficult to be mass-produced, the product iscomplex in configuration, it will take one or two hours to assemble theproduct successfully by a professional skilled person.

SUMMARY

The present disclosure provides a method for aligning a coordinate of acontroller or a headset with a coordinate of a binocular system to aligna positive direction of the controller or the headset with a positivedirection of a virtual environment coordinate system.

According to one aspect of the disclosure, a method for aligning acoordinate of a controller with a coordinate of a binocular systemincludes: obtaining pose data of the controller; acquiring real-timecoordinate data of the controller in a binocular coordinate systemduring a repeating movement based on a light spot emitted from thecontroller; generating a motion track of the controller during therepeating movement based on the real-time coordinate data and the posedata corresponding to the real-time coordinate data; determining a poseof the controller in a controller coordinate system based on the motiontrack, and the pose being defined as a positive direction of thecontroller in the binocular coordinate system; determining an anglebetween the positive direction of the controller coordinate system and apositive direction of the binocular coordinate system, the angle beingdefined as a first angle; and correcting the binocular coordinate systembased on the first angle until the angle between the positive directionof the controller and the positive direction of the binocular coordinatesystem reaches zero.

According to another aspect of the disclosure, a method for aligning acoordinate of a headset with a coordinate of a binocular systemincludes: estimating a position and orientation of the headset in thebinocular coordinate system based on two light spots emitted from theheadset, wherein the orientation of the headset in the binocularcoordinate system is defined as a headset direction vector; calculatingan angle between the headset direction vector and a preset binoculardirection vector in the binocular coordinate system, the angle isdefined as a second angle; adjusting an angle between a positivedirection of an object corresponding to the headset in the virtualenvironment and the positive direction of the virtual environmentcoordinate system of the virtual environment based on the second angle;and adjusting the position of the object corresponding to the headset inthe virtual environment in the binocular coordinate system based on theposition of the headset in the binocular coordinate system.

According to another aspect of the disclosure, a device for aligning acoordinate of a controller with a coordinate of a binocular systemincludes one or more processors; and memory storing one or more programsconfigured to be executed by the one or more processors, the one or moreprograms comprising instructions for: obtaining pose data of thecontroller; acquiring real-time coordinate data of the controller in abinocular coordinate system during a repeating movement based on a lightspot emitted from the controller; generating a motion track of thecontroller during the repeating movement based on the real-timecoordinate data and the pose data corresponding to the real-timecoordinate data; determining a pose of the controller in a controllercoordinate system based on the motion track, and the pose being definedas a positive direction of the controller in the binocular coordinatesystem; determining an angle between the positive direction of thecontroller coordinate system and a positive direction of the binocularcoordinate system, the angle being defined as a first angle; andcorrecting the binocular coordinate system based on the first angleuntil the angle between the positive direction of the controller and thepositive direction of the binocular coordinate system reaches zero.

Based on the above embodiment, when the IMU is aligned to thecontroller, only one light spot is needed to be arranged on thecontroller. The alignment between the positive direction of thecontroller (or the headset) and the positive direction of the virtualenvironment coordinate system is realized. The implementation process issimple, and the cost is low.

BRIEF DESCRIPTION OF THE DRAWINGS

To describe the technical solutions in the embodiments of the presentdisclosure more clearly, the following briefly introduces theaccompanying drawings required for describing the embodiments of thepresent disclosure or the prior art. Apparently, the accompanyingdrawings in the following description show merely some embodiments ofthe present disclosure, and a person of ordinary skill in the art maystill derive other drawings from these accompanying drawings withoutcreative efforts.

FIG. 1 is a flowchart of an embodiment of a method for aligning acoordinate of a controller with a coordinate of a binocular systemaccording to the present disclosure.

FIG. 2 is a flowchart of another embodiment of a method for aligningcoordinate of a headset with a coordinate of the binocular systemaccording to the present disclosure.

FIG. 3 is a flowchart of an embodiment of calculating an angle between apositive direction of a controller and a positive direction of abinocular coordinate system according to the present disclosure.

FIG. 4 is a schematic diagram of an embodiment of a device for aligningthe coordinate of the controller with the coordinate of the binocularsystem according to one embodiment of the present disclosure.

DETAILED DESCRIPTION

It should be understood at the outset that although an illustrativeimplementation of one or more embodiments are provided below, thedisclosed systems and/or methods may be implemented using any number oftechniques, whether currently known or in existence. The disclosureshould in no way be limited to the illustrative implementations,drawings, and techniques illustrated below, including the exemplarydesigns and implementations illustrated and described herein, but may bemodified within the scope of the appended claims along with their fullscope of equivalents.

The present disclosure discloses a method for aligning a coordinate of acontroller with a coordinate of a binocular system, which may be appliedon a VR device, an AR device, or an MR device. FIG. 1 illustrates aflowchart of a method for aligning the coordinate of the controller withthe coordinate of the binocular system. The method includes thefollowing blocks.

At block S101, sensor data of the controller can be acquired.

The controller may be a remote-controller, such as a hand-holdcontroller of virtual reality device, augmented reality device or mixedreality device. A plurality of controllers can be arranged in a set ofdevices. In general, it is common that two controllers are provided in aset of devices. An IMU (Inertial Measurement Unit) is arranged insidethe controller. The IMU includes a plurality of sensors, which are usedfor measuring a pose of the controller. The sensor data obtained by thesensors may be fused to obtain a pose data O1 (yaw, pitch and roll), andO2 (yaw, pitch, roll) of the two controllers, respectively.

At block S102, pose data is obtained based on the sensor data.

At block S103, real-time coordinate data of the controller in thebinocular coordinate system during a repeating movement can be acquiredbased on a light spot emitted from the controller, while the pose dataof the controller is obtained.

A light spot emitting unit arranged on each controller is used to emit alight spot with an identity. The light spot is monitored and tracked,via a light spot tracking unit, to obtain the real-time coordinate datain the binocular coordinate system during the repeating movement. Eachreal-time coordinate data corresponds to a pose data of the controller.A motion track of the controller in the binocular coordinate system canbe obtained by processing the real-time coordinate data and the posedata, while the controller moves in the repeating movement.

At block S104, the motion track of the controller during the repeatingmovement can be generated based on the real-time coordinate data and thepose data corresponding to the real-time coordinate data.

At block S105, a pose of the controller in a controller coordinatesystem can be determined based on the motion track, and the pose isdefined as a positive direction of the controller in the binocularcoordinate system.

The positive direction of the controller in the binocular coordinatesystem refers to a facing orientation of the controller. In oneembodiment, the positive direction of the controller in the binocularcoordinate system is determined based on the pose data and the motiontrack of the controller. The positive direction of the controller in thebinocular coordinate system is defined as the positive direction of thecontroller.

At block S106, an angle between the positive direction of the controllercoordinate system and a positive direction of the binocular coordinatesystem can be determined according to the pose of the controller and thereal-time coordinate data of the controller, and the angle can bedefined as a first angle.

In some embodiments, a plurality of the first angles can be obtained,when a plurality of controllers are provided. The coordinate system maybe confused, if adjusts the binocular coordinate system whenever onefirst angel is obtained. Therefore, before adjusting the binocularcoordinate system, an average value of the plurality of the first angelscan be obtained. The average value is regarded as the first angle foradjusting or correcting coordinates in the binocular coordinate systemin subsequent blocks.

At block S107, the binocular coordinate system can be corrected based onthe first angle until the angle between the positive direction of thecontroller and the positive direction of the binocular coordinate systemreaches zero, and the positive direction of a virtual objectcorresponding to the controller in a virtual environment is aligned withthe positive direction of the virtual environment coordinate system.

In some embodiments, a preset coordinate mapping relationship is setbetween the virtual environment coordinate system and the binocularcoordinate system in a virtual reality device, an augmented realitydevice or a mixed reality device. When the coordinate of one of thevirtual environment coordinate system and the binocular coordinatesystem is changed, the coordinate of the other coordinate system mayalso be changed. In this embodiment, the positive direction of thebinocular coordinate system is adjusted based on the first angle, whenthe first angle between the positive direction of the controller and thepositive direction of the binocular coordinate system is obtained, sothat the positive direction of the binocular coordinate system isaligned with the positive direction of the virtual reality device,augmented reality device or the mixed reality device. At the same time,the virtual environment coordinate system will be changed based on thebinocular coordinate system, thus the positive direction of the virtualobject corresponding to the controller in the virtual environment isaligned with the positive direction of the virtual environment.

Wherein, the binocular coordinate system has a preset mappingrelationship with the virtual environment coordinate system.

According to the disclosed embodiment of the present disclosure, onlyone light spot is needed to be arranged on the controller when the IMUis aligned with the controller. Thus, an alignment between the positivedirection of the controller and the positive direction of the virtualenvironment coordinate system is realized, which has the advantages ofsimple process and lower cost.

In order to obtain the first angle with higher precision by calculating,the repeating movement in the embodiment of the present disclosure mayrefer to a repeating movement of a regular track, for example, therepeating movement of the controller can be in a linear reciprocatingmovement or a circular looping movement of the controller. The blockS103 may be further defined as the following steps.

The real-time coordinate data in the binocular coordinate system of thecontroller can be acquired based on the light spot emitted from thecontroller during the linear reciprocating movement or the circularlooping movement of the controller.

In addition, in order to further improve the accuracy of the firstangle, the present disclosure further discloses a specific process forobtaining the first angle. Referred to FIG. 3, the block S105 and blockS106 further includes the following steps.

At block S301, the motion track of the controller during the movementcan be acquired, wherein the motion track includes the real-timecoordinate data and the pose data of the controller.

At block S302, the motion track which is straight-line or circle-likecan be judged by performing pattern recognition, a block S303 can beexecuted when the motion track is straight-line, and a block S305 can beexecuted when the movement track is circle-like.

At the block S303, a linear fitting can be performed on the motion trackwhen the motion track is straight-line, and a block S304 can beexecuted.

At the block S304, a linear direction perpendicular to the motion tracklinear fitted can be obtained by calculating.

At the block S305, a circle fitting can be performed on the motion trackwhen the motion track is circle-like, and a block S306 can be executed.

At the block S306, a linear direction passing through the center of themotion track circle fitted can be obtained by calculating.

At block S307, a slope K of the linear direction in the binocularcoordinate system can be obtained by calculating.

Wherein the slope K refers to a slope relative to the positive directionof the binocular coordinate system in a plane formed by a verticaldirection of the slope K and the positive direction of the binocularcoordinate system.

At block S308, an angle θ between the linear direction and the positivedirection of the binocular coordinate system can be obtained based onthe slope K.

At block S309, the motion track is segmented to obtain a plurality ofsub-motion tracks, and the slopes K_(i) of linear directioncorresponding to each of the sub-motion tracks in the binocularcoordinate system can be obtained by calculating, wherein the i of K_(i)corresponds to each of the sub-motion tracks.

In some embodiments, in order to ensure the accuracy of the calculatingresult, the angle θ is further corrected. The motion track can besegmented by a preset distance. The preset distance may refer to thedistance between two sampling spots on the motion track, that is, themotion track between each two sampling spots is defined as thesub-motion track. The sub-motion tracks obtained by segmenting areprocessed based on the block S302 to the block S308. The slope K_(i)corresponding to each of the sub-motion tracks is obtained.

At block S310, a similarity D_(i) between the slope K_(i) correspondingto each of the sub-motion tracks and the slope K can be obtained bycalculating.

At block S311, an weighted average horizontal angle δ can be obtainedbased on a formula δ=Σ_(i=1) ^(N)K_(i)*D _(i) (1), wherein the angle δrefers to an angle between the natural direction of the controller and avertical direction of a binocular plane which is also defined as anazimuth angle, and the angle δ may be obtained by the pose data, N is atotal number of the plurality of sub-motion tracks.

At block S312, the sum of the angle θ and δ can be the angle between thepositive direction of the controller and the positive direction of thebinocular coordinate system.

In addition to provide a method for aligning the positive direction ofthe controller with the positive direction of the binocular system(namely the virtual environment coordinate system), the presentdisclosure further provides a method for aligning a coordinate of aheadset and a coordinate of the binocular system. As shown in FIG. 2,the method may include the following steps.

At block S201, the position and orientation of the headset in thebinocular coordinate system can be estimated based on two light spotsemitted from the headset, and the orientation of the headset in thebinocular coordinate system is defined as a headset direction vector.

The headset is provided with two light spot emitting unit. The two lightspot emitting unit is used to emit two light spots. The position anddirection of the two light spots in the binocular coordinate can beestimated when the two light spots are detected by a light spot trackingunit, wherein the position and direction may be an orientationcorresponding to the two light spots, and the orientation of the twolight spots of the headset in the binocular coordinate system is definedas the headset direction vector.

At block S202, an angle between the headset direction vector and apreset binocular direction vector in the binocular coordinate system canbe obtained by calculating, and the angle is defined as a second angle.

At block S203, an angle between a positive direction of an objectcorresponding to the headset in the virtual environment and the positivedirection of the virtual environment coordinate system can be adjustedbased on the second angle.

In some embodiments, the object corresponding to the headset in thevirtual environment may be considered as a virtual user. The virtualuser may be adjusted, based on the second angle, so as to make the anglebetween the positive direction of the virtual user and the positivedirection of the virtual environment coordinate system equal to thesecond angle. Thus, the orientation of the headset is aligned in thevirtual environment coordinate system.

At block S204, the position of the object of the virtual environmentcorresponding to the headset in the binocular coordinate system can beadjusted based on the position of the headset in the binocularcoordinate system, and the position of the headset in the virtualenvironment coordinate system can be aligned.

In another embodiment of the present disclosure, the method ofestimating the position and orientation of the headset in the binocularcoordinate system based on the two light spots emitted from the headsetincludes the following.

The position of the two light spots in the binocular coordinate systemcan be obtained by processing the image of the two light spots emittedfrom the headset, with stereo measurement estimation. The position ofthe headset in the binocular coordinate system can be obtained bycalculating the position of the two light spots in the binocularcoordinate system, and the position of the virtual user corresponding tothe headset in the virtual environment coordinate system can beobtained. The positive direction of the two light spots in the binocularcoordinate system can be obtained by calculating the position of the twolight spots in the binocular coordinate system. The positive directionof the two light spots is defined as the positive direction of theheadset in the binocular coordinate system. Thus, the positive directionof the virtual user corresponding to the headset in the virtualenvironment coordinate system is obtained.

The disclosure also provides a device for aligning a coordinate of acontroller with a coordinate of a binocular system corresponding to themethod mentioned above. As shown in FIG. 4, the device for aligningcoordinate of a coordinate of a controller with a coordinate of abinocular system may be applied on a VR device, an AR device, or an MRdevice. The device includes a controller direction alignment subsystem100. The controller direction alignment subsystem 100 is used to finallyalign the virtual object corresponding in the controller in the virtualenvironment with the virtual environment coordinate system.

The controller direction alignment subsystem 100 may include acontroller pose calculating unit 101, a motion track calculating unit102, a first angle calculating unit 103, and a binocular coordinateadjusting unit 104.

The controller pose calculating unit 101 is corresponding to the blocksS101 and S102 of the method for aligning the coordinate of thecontroller with the coordinate of the binocular system. The controllerpose calculating unit 101 is used for acquiring sensor data of thecontroller and obtaining pose data according to the sensor data of thecontroller.

The motion track calculating unit 102 is corresponding to the blocksS103 and S104 of the method for aligning coordinates. The motion trackcalculating unit 102 is used for acquiring real-time coordinate data ofthe controller in the binocular coordinate system during a repeatingmovement based on a light spot emitted from the controller andgenerating a motion track of the controller in the movement based on thereal-time coordinate data and the pose data corresponding to thereal-time coordinate data.

The first angle calculating unit 103 is corresponding to the blocks S105and S106 of the corresponding method for aligning coordinates. The firstangle calculating unit 103 is used for determining the pose of thecontroller in a controller coordinate system based on the motion track,wherein the positive direction of the controller in the binocularcoordinate system is defined as the positive direction of thecontroller, and determining an angle between the positive direction ofthe controller coordinate system and a positive direction of thebinocular coordinate system by calculating, wherein the angle is definedas the first angle.

The binocular coordinate adjusting unit 104 is corresponding to theblock S107. The binocular coordinate adjusting unit 104 is used forcorrecting the binocular coordinate system based on the first angleuntil an angle between the positive direction of the controller and thepositive direction of the binocular coordinate system reaches zero, sothat the positive direction of the virtual object corresponding to thecontroller in a virtual environment is aligned with the positivedirection of the binocular coordinate system, wherein a presetcoordinate mapping relationship is established between the binocularcoordinate system and the virtual environment coordinate system.

Corresponding to the method for aligning coordinates above, the firstangle calculating unit 103 may be configured for: acquiring the motiontrack of the controller during the movement; when the motion track is astraight line, the motion track is subjected to a linear fitting; astraightness of the motion track is obtained by calculation, and alinear direction perpendicular to the motion track which is linearfitted can be obtained by calculating; when the motion track iscircle-like, the motion track is subjected to a circle fitting; acurvature of the motion track is obtained by calculation, and a lineardirection passing through a center of the motion track is obtained bycalculation; obtaining a slope K of the linear direction in thebinocular coordinate system; obtaining an angle θ between the lineardirection and the positive direction of the binocular coordinate systembased on the slope K; segmenting the motion track to obtain a pluralityof sub-motion tracks and obtaining the slopes K_(i) of linear directioncorresponding to each of the sub-motion tracks in the binocularcoordinate system by calculating, wherein the i corresponds to each ofthe sub-motion tracks; obtaining a similarity D_(i) between the slopeK_(i) corresponding to each of the sub-motion tracks and the slope K;obtaining a weighted average horizontal angle δ based on a formulaδ=Σ_(i=1) ^(N)K_(i)*D _(i), wherein N is a total number of the pluralityof sub-motion tracks; obtaining a sum of the angle θ and the angle δ,wherein the sum of the angle θ and the angle δ can be the angle betweenthe positive direction of the controller and the positive direction ofthe binocular coordinate system.

Corresponding to the method for aligning the coordinate of the headsetwith the coordinate of the binocular system, the device for aligning thecoordinate of the controller with the coordinate of the binocular systemfurther includes a subsystem for aligning the coordinate of the headsetwith the coordinate of the binocular system.

The headset direction alignment subsystem 200 includes a headsetposition and direction calculating unit 201, a second angle calculatingunit 202, and a virtual user adjusting unit 203.

The headset position and direction calculating unit 201 is correspondingto the block S201 of the method. The headset is used for estimating aposition and an orientation of the headset in the binocular coordinatesystem based on the two light spots emitted from the headset anddefining the direction of the headset in the binocular coordinate systemas a headset direction vector.

The second angle calculating unit 202 is corresponding to the block S202of the method. The second angle calculating unit 202 is used forcalculating an angle between the headset direction vector and apredetermined binocular direction vector in the binocular coordinatesystem and defining the angel as a second angle.

The virtual user adjusting unit 203 is corresponding to the blocks S203and S204 of the method for aligning coordinates. The virtual useradjusting unit 203 is used for adjusting an angle between a positivedirection of the object corresponding to the headset in the virtualenvironment and the positive direction of the virtual environmentcoordinate system based on the second angle, and also used for adjustinga position of the object of the virtual environment corresponding to theheadset in the binocular coordinate system based on the position of theheadset in the binocular coordinate system.

Corresponding to the method for aligning the coordinate of thecontroller with the coordinate of the binocular system, the motion trackcalculating unit 102 is used for acquiring real-time coordinate data inthe binocular coordinate system of the controller in a reciprocatingmovement or a circular looping movement based on the light spots emittedfrom the controller and generating the motion track of the controller ina movement based on the real-time coordinate data.

Corresponding to the method, the headset position and directioncalculating unit 201 is configured for: processing the image of the twolight spots emitted from the headset to obtain the position of the twolight spots in the binocular coordinate system by stereo measurementestimating, calculating the position of the headset in the binocularcoordinate system based on the position of the two light spots in thebinocular coordinate system, calculating a positive direction of the twolight spots in the binocular coordinate system based on the position ofthe two light spots in the binocular coordinate system.

Corresponding to the system for aligning coordinates, the disclosurefurther provides a virtual reality system, an augmented reality system,or a mixed reality system. The system may include a head mounteddisplay, a controller, a two light spot emitting unit, a light spotemitting unit and a light spot tracking unit. All the units in thesystem for aligning coordinates disclosed by the embodiment are arrangedin corresponding device respectively.

The two light spot emitting unit is coupled to the head mounted display.The two light spot emitting unit is used for emitting two light spotswith a unique ID identity respectively.

The controller is communicated with the head mounted display in awireless manner or in a wired manner.

The light spot emitting unit coupled to the controller and is used foremitting one light spot with a unique ID identity.

The light spot tracking unit is used for capturing the light spots. Thelight spot tracking unit includes the motion track calculating unit andthe headset position and direction calculating unit. The light spottracking unit is used to obtain a motion track and a headset directionvector sent to the headset via the two light spot emitting unit.

The headset includes the controller pose calculating unit, the firstangle calculating unit, the binocular coordinate adjusting unit, thesecond angle calculating unit and the virtual user adjusting unit.

In use, the two light spot emitting unit is installed on the headset.When the two light spot emitting unit is a non-integrated headset, amobile phone is inserted into the headset, and the headset is arrangedin front of a binocular camera (also viewed as a binocular trackingunit). The headset is worn on the head. The binocular tracking unit cansee two light spots emitted by the two light spot emitting unit. Each ofthe two light spots is provided with an ID, so that a light spot P1 canbe distinguished from a light spot P2. The binocular tracking unitestimates a three-dimensional measurement of image processing on thelight spots to obtain the position P1 (x, y, z) and P2 (x, y, z) of thetwo light spots in the binocular coordinate system. The position of thetwo light spots is transmitted to the two light spot emitting unit by awireless or wired mode, and then and the position of the two light spotsis sent to a processor of the headset for algorithm calculating. Afterthe headset obtains the position of the two light spots, the positivedirection of the two light spot is estimated, and the positive directionof the two light spots is the positive direction of the head mounteddisplay. The display center is placed in the positive direction of thebinocular camera so as to locate the direction of the two light spots inthe binocular coordinate system and form a direction vector (headsetdirection vector). The direction vector is used for obtaining the anglebetween the headset direction vector and the direction vector of thepositive direction of the binocular coordinate system by calculating.That is, the position and the direction of the virtual usercorresponding to the headset in the virtual environment can be directlyobtained through an angle correction.

The process of aligning the controller is shown in the followingembodiment.

The headset is connected to two handles through the Bluetooth. Eachhandle represents a controller. The handle is used for obtaining dataand fusing data from the sensor through an internal calculating unit MCU(Microprogrammed Control Unit), so as to obtain a handle pose data O1(yaw, pitch, roll) and O2 (yaw, pitch, roll). The handle pose data istransmitted to the headset through the Bluetooth or a data line. Thelight spots of the handle are identified and positioned by the lightspot tracking unit to obtain coordinates of the handle in a binocularcoordinate system, that is, P3 (X, Y, Z) and P4 (X, Y, Z). Then, a usertakes up the handle and performs transversely or rotationally repeatingmotion in front of the binocular tracking unit. The binocular trackingunit obtains the light spots of the left handle and the right handlebased on an image color by an image processing and stereoscopic vision.The 3D position of the light spots emitted from the handle in thebinocular coordinate system is obtained through three-dimensionalvision. The 3D position information of the light spots each representinga corresponding handle in the binocular coordinate system is sent to theheadset through the light spot emitting unit via wireless manner or datalines. The handles and the headset are synchronized through time stampsto obtain 6DOF data on the unified time axis. After the headset receivesthe position and the pose data of the handle, the angle between thepositive direction of the handle and the positive direction of thebinocular coordinate system is estimated based on the motion track ofthe handle, such as left and right linear motion or rotating circularmotion. The positive direction of the binocular coordinate system isadjusted based on the angle. The positive direction of the handle isaligned with the positive direction of the binocular coordinate system,namely, in the virtual environment, the positive direction of the handor the hand-held object is aligned with the positive direction of thevirtual environment. A calculating unit is arranged inside the headset,and the built-in algorithm is used for detecting the motion track of thelight spot of the handle. The direction of the motion track of thehandle can be estimated by restraining the movement through the IMUdata. The azimuth angle of the handle (the angle between the positivedirection of the handle and the positive direction of the binocularcoordinate system) is obtained. The positive direction of the handle isaligned with the positive direction of the binocular coordinate systemthrough the azimuth angle, and the positive direction of the binocularcoordinate system is also the direction of the person in the virtualenvironment.

For convenience of description, the functions of the system aredescribed as various modules respectively. Of course, the functions ofthe modules can be implemented in one or more software and/or hardwarewhen implementing the present invention.

Various embodiments in the specification are described in a progressivemanner, and the parts with the same similarity among the embodiments canbe seen from each other. The emphasis instead is placed upon clearlyillustrating the differences between the present disclosure. For systemsor system embodiments, it is basically like the method embodiments, sothat the description is simple and relevant to the description of themethod embodiment. The system and system embodiments described above aremerely schematic, wherein the units described as separate parts may ormay not be physically separated. The component displayed as a unit canbe or may not be a physical unit or can be distributed to a plurality ofnetwork units. The objective of the embodiment can be realized byselecting some or all the modules according to actual requirements.Persons of ordinary skilled in the art can understand and implement themethod without creative efforts.

The professional personnel can further realize the steps of the unitsand algorithm steps described by the embodiments of the presentdisclosure can be implemented in an electronic hardware, computersoftware or a combination of the two. In order to clearly illustrate theinterchangeability of hardware and software, in the above description,the composition and the steps of the examples have been describedgenerally in terms of functions. Whether the functions are executed byhardware or software depends on the specific application and designconstraints of the technical scheme. A person skilled in the art can usedifferent methods to implement the described functions for eachapplication, but such implementation should not be beyond the scope ofthe present disclosure.

The steps of the method or algorithm described in connection with theembodiments disclosed herein may be embodied directly in hardware, asoftware module executed by a processor, or a combination of thesoftware modules and the software modules executed by the processor. Thesoftware module may be placed in a random-access memory (RAM), a memory,a read-only memory (ROM), an electrically programmable ROM, anelectrically erasable programmable ROM and a register, a hard disk, aremovable magnetic disk, a CD-ROM, or any other form of storage mediumknown in the technical field.

It should also be noted that, in the present disclosure, such as firstand second, are only used to distinguish one entity or operation fromanother entity or operation, and any actual relationship or sequenceexists between the entities or the operations does not need to berequired or implied. Moreover, the terms “comprising”, “include” or anyother variants thereof are intended to encompass a non-exclusiveinclusion, so that the process and the method which comprise a series ofelements are achieved. The article or the device not only includes thoseelements but also includes other elements not explicitly listed, orfurther comprises the inherent elements of the process, the method, thearticle or the equipment. In the case that there are no morerestrictions, the statement “comprising one” is not excluded from theprocess comprising the element, the method, the article or the equipmentfurther have the same elements in the article or the equipment.

The above description of the disclosed embodiments can enable any personskilled in the art to realize or use the present disclosure. Variousmodifications to these embodiments will be apparent to those skilled inthe art, the general principles defined herein may be practiced withoutdeparting from the spirit or scope of the present disclosure in otherembodiments. Accordingly, the disclosure will not be limited to theembodiments shown herein but is to be accorded the widest scopeconsistent with the principles and novel features disclosed herein.

What is claimed is:
 1. A method for aligning a coordinate of acontroller with a coordinate of a binocular system, comprising:obtaining pose data of the controller; acquiring real-time coordinatedata of the controller in a binocular coordinate system during arepeating movement based on a light spot emitted from the controller;generating a motion track of the controller during the repeatingmovement based on the real-time coordinate data and the pose datacorresponding to the real-time coordinate data; determining a pose ofthe controller in a controller coordinate system based on the motiontrack, and the pose being defined as a positive direction of thecontroller in the binocular coordinate system; determining an anglebetween the positive direction of the controller coordinate system and apositive direction of the binocular coordinate system, the angle beingdefined as a first angle, comprising: acquiring the motion track of thecontroller in the movement; when the motion track is circle-like, themotion track is subjected to a circle fitting, and a linear directionpassing through a center of the motion track circle fitted is obtainedby calculation; obtaining a slope K of the linear direction in thebinocular coordinate system by calculation; obtaining an angle θ betweenthe linear direction and the positive direction of the binocularcoordinate system based on the slope K by calculation; segmenting themotion track to obtain a plurality of sub-motion tracks and obtainingthe slopes K_(i) of a linear direction corresponding to each of thesub-motion tracks in the binocular coordinate system, wherein icorresponds to each of the sub-motion tracks; obtaining a similarityD_(i) between the slope K_(i) corresponding to each of the sub-motiontracks and the slope K by calculation; obtaining a weighted averagehorizontal angle δ based on a formula δ=Σ_(i=1) ^(N)K_(i)*D _(i) whereinN is a total number of the plurality of sub-motion tracks; and obtaininga sum of the angle θ and the angle δ to be an angle between the positivedirection of the controller and the positive direction of the binocularcoordinate system; and correcting the binocular coordinate system basedon the first angle until the angle between the positive direction of thecontroller and the positive direction of the binocular coordinate systemreaches zero.
 2. The method of claim 1, further comprising: acquiringsensor data of the controller; and obtaining the pose data of thecontroller according to the sensor data.
 3. The method of claim 1,wherein correcting the binocular coordinate system further comprises:aligning a positive of a virtual object corresponding to the controllerin a virtual environment with a positive direction of a coordinate ofthe virtual environment, wherein a preset coordinate mappingrelationship is established between the binocular coordinate system andthe virtual environment coordinate system.
 4. The method of claim 1,further comprising: estimating a position and orientation of a headsetin the binocular coordinate system based on two light spots emitted fromthe headset, wherein the orientation of the headset in the binocularcoordinate system is defined as a headset direction vector; obtaining anangle between the headset direction vector and a preset binoculardirection vector in the binocular coordinate system, the angle isdefined as a second angle; adjusting an angle between a positivedirection of an object corresponding to the headset in the virtualenvironment and the positive direction of the virtual environmentcoordinate system of the virtual environment based on the second angle;and adjusting the position of the object corresponding to the headset inthe virtual environment in the binocular coordinate system based on theposition of the headset in the binocular coordinate system.
 5. Themethod of claim 4, wherein estimating the position and the direction ofthe headset comprises: obtaining the position of the two light spots inthe binocular coordinate system by processing an image of the two lightspots emitted from the headset with stereo measurement estimation;obtaining the position of the headset in the binocular coordinate systembased on the position of the two light spots in the binocular coordinatesystem; obtaining a positive direction of the two light spots in thebinocular coordinate system based on the position of the two light spotsin the binocular coordinate system; and defining the positive directionof the two light spots as the positive direction of the headset.
 6. Themethod of claim 1, wherein acquiring the real-time coordinate data inthe binocular coordinate system in the repeating movement of a regulartrack of the controller is based on the light spot emitted from thecontroller.
 7. The method of claim 6, wherein acquiring the real-timecoordinate data in the binocular coordinate system in the repeatingmovement of a regular track of the controller comprises: acquiring thereal-time coordinate data in the binocular coordinate system either in alinear reciprocating movement of the controller or a circular loopingmovement of the controller based on the light spot emitted from thecontroller.
 8. A method for aligning a coordinate of a controller with acoordinate of a binocular system, comprising: obtaining pose data of thecontroller; acquiring real-time coordinate data of the controller in abinocular coordinate system during a repeating movement based on a lightspot emitted from the controller; generating a motion track of thecontroller during the repeating movement based on the real-timecoordinate data and the pose data corresponding to the real-timecoordinate data; determining a pose of the controller in a controllercoordinate system based on the motion track, and the pose being definedas a positive direction of the controller in the binocular coordinatesystem; determining an angle between the positive direction of thecontroller coordinate system and a positive direction of the binocularcoordinate system, the angle being defined as a first angle, comprising:acquiring the motion track of the controller in the movement; whereinwhen the motion track is a straight line, the motion track is subjectedto a linear fitting, and a linear direction perpendicular to the motiontrack linear fitted is obtained by calculating; obtaining a slope K ofthe linear direction in the binocular coordinate system by calculation;obtaining an angle θ between the linear direction and the positivedirection of the binocular coordinate system based on the slope K bycalculation; segmenting the motion track to obtain a plurality ofsub-motion tracks and obtaining the slopes K₁ of a linear directioncorresponding to each of the sub-motion tracks in the binocularcoordinate system, wherein i corresponds to each of the sub-motiontracks; obtaining a similarity D_(i) between the slope K_(i)corresponding to each of the sub-motion tracks and the slope K bycalculation; obtaining a weighted average horizontal angle δ based on aformula δ=Σ_(i=1) ^(N)K_(i)*D _(i) wherein N is a total number of theplurality of sub-motion tracks; and obtaining a sum of the angle θ andthe angle δ to be an angle between the positive direction of thecontroller and the positive direction of the binocular coordinatesystem; and correcting the binocular coordinate system based on thefirst angle until the angle between the positive direction of thecontroller and the positive direction of the binocular coordinate systemreaches zero.
 9. The method of claim 8, wherein a straightness of themotion track is obtained when the motion track is a straight line. 10.The method of claim 1, wherein a curvature of the motion track isobtained when the motion track is circle-like.
 11. A device for aligninga coordinate of a controller with a coordinate of a binocular system,comprising: one or more processors; and a memory storing one or moreprograms configured to be executed by the one or more processors, theone or more programs comprising instructions for: obtaining pose data ofthe controller; acquiring real-time coordinate data of the controller ina binocular coordinate system during a repeating movement based on alight spot emitted from the controller; generating a motion track of thecontroller during the repeating movement based on the real-timecoordinate data and the pose data corresponding to the real-timecoordinate data; determining a pose of the controller in a controllercoordinate system based on the motion track, and the pose being definedas a positive direction of the controller in the binocular coordinatesystem; determining an angle between the positive direction of thecontroller coordinate system and a positive direction of the binocularcoordinate system, the angle being defined as a first angle, comprising:acquiring the motion track of the controller in the movement; when themotion track is circle-like, the motion track is subjected to a circlefitting, and a linear direction passing through a center of the motiontrack circle fitted is obtained by calculation; obtaining a slope K ofthe linear direction in the binocular coordinate system by calculation;obtaining an angle θ between the linear direction and the positivedirection of the binocular coordinate system based on the slope K bycalculation; segmenting the motion track to obtain a plurality ofsub-motion tracks and obtaining the slopes K_(i) of a linear directioncorresponding to each of the sub-motion tracks in the binocularcoordinate system, wherein i corresponds to each of the sub-motiontracks; obtaining a similarity D_(i) between the slope K_(i)corresponding to each of the sub-motion tracks and the slope K bycalculation; obtaining a weighted average horizontal angle δ based on aformula δ=Σ_(i=1) ^(N)K_(i)*D _(i), wherein N is a total number of theplurality of sub-motion tracks; and obtaining a sum of the angle θ andthe angle δ to be an angle between the positive direction of thecontroller and the positive direction of the binocular coordinatesystem; and correcting the binocular coordinate system based on thefirst angle until the angle between the positive direction of thecontroller and the positive direction of the binocular coordinate systemreaches zero.
 12. The device of claim 11, wherein the one or moreprograms further comprising instructions for: acquiring sensor data ofthe controller; and obtaining the pose data of the controller accordingto the sensor data.
 13. The device of claim 11, wherein correcting thebinocular coordinate system comprises aligning a positive of a virtualobject corresponding to the controller in a virtual environment with apositive direction of a coordinate of the virtual environment, wherein apreset coordinate mapping relationship is established between thebinocular coordinate system and the virtual environment coordinatesystem.
 14. The device of claim 11, wherein the one or more programsfurther comprising instructions for: estimating a position andorientation of a headset in the binocular coordinate system based on twolight spots emitted from the headset, wherein the orientation of theheadset in the binocular coordinate system is defined as a headsetdirection vector; obtaining an angle between the headset directionvector and a preset binocular direction vector in the binocularcoordinate system, the angle is defined as a second angle; adjusting anangle between a positive direction of an object corresponding to theheadset in the virtual environment and the positive direction of thevirtual environment coordinate system of the virtual environment basedon the second angle; and adjusting the position of the objectcorresponding to the headset in the virtual environment in the binocularcoordinate system based on the position of the headset in the binocularcoordinate system.
 15. The device of claim 14, wherein estimating theposition and the direction of the headset in the binocular coordinatesystem comprises: obtaining the position of the two light spots in thebinocular coordinate system by processing an image of the two lightspots emitted from the headset with stereo measurement estimation;obtaining the position of the headset in the binocular coordinate systembased on the position of the two light spots in the binocular coordinatesystem; obtaining a positive direction of the two light spots in thebinocular coordinate system based on the position of the two light spotsin the binocular coordinate system; and defining the positive directionof the two light spots as the positive direction of the headset.
 16. Thedevice of claim 11, wherein determining the angle between the controllercoordinate system and the binocular coordinate system comprises:acquiring the motion track of the controller in the movement; whereinwhen the motion track is a straight line, the motion track is subjectedto a linear fitting, and a linear direction perpendicular to the motiontrack linear fitted is obtained by calculating; obtaining a slope K ofthe linear direction in the binocular coordinate system by calculation;obtaining an angle θ between the linear direction and the positivedirection of the binocular coordinate system based on the slope K bycalculation; segmenting the motion track to obtain a plurality ofsub-motion tracks and obtaining the slopes K_(i) of a linear directioncorresponding to each of the sub-motion tracks in the binocularcoordinate system, wherein i corresponds to each of the sub-motiontracks; obtaining a similarity D_(i) between the slope K_(i)corresponding to each of the sub-motion tracks and the slope K bycalculation; obtaining a weighted average horizontal angle δ based on aformula δ=Σ_(i=1) ^(N)K_(i)*D _(i) wherein N is a total number of theplurality of sub-motion tracks; and obtaining a sum of the angle θ andthe angle δ to be an angle between the positive direction of thecontroller and the positive direction of the binocular coordinatesystem.
 17. The device of claim 11, wherein acquiring the real-timecoordinate data in the binocular coordinate system is in the repeatingmovement of a regular track of the controller, based on the light spotemitted from the controller.
 18. The device of claim 17, whereinacquiring the real-time coordinate data in the binocular coordinatesystem in the repeating movement of a regular track of the controllercomprises: acquiring the real-time coordinate data in the binocularcoordinate system either in a linear reciprocating movement of thecontroller or a circular looping movement of the controller based on thelight spot emitted from the controller.
 19. The device of claim 14,wherein the controller is communicated with the headset in a wirelessmanner or in a wired manner.
 20. The method of claim 1, whereinacquiring the real-time coordinate data in the binocular coordinatesystem is in the repeating movement of a regular track of thecontroller, based on the light spot emitted from the controller.